Towards extending the SWITCH platform for time-critical, cloud-based CUDA applications: Job scheduling parameters influencing performance

SWITCH (Software Workbench for Interactive, Time Critical and Highly self-adaptive cloud applications) allows for the development and deployment of real-time applications in the cloud, but it does not yet support instances backed by Graphics Processing Units (GPUs). Wanting to explore how SWITCH might support CUDA (a GPU architecture) in the future, we have undertaken a review of time-critical CUDA applications, discovering that run-time requirements (which we call ‘wall time’) are in many cases regarded as the most important. We have performed experiments to investigate which parameters have the greatest impact on wall time when running multiple Amazon Web Services GPU-backed instances. Although a maximum of 8 single-GPU instances can be launched in a single Amazon Region, launching just 2 instances rather than 1 gives a 42% decrease in wall time. Also, instances are often wasted doing nothing, and there is a moderately-strong relationship between how problems are distributed across instances and wall time. These findings can be used to enhance the SWITCH provision for specifying Non-Functional Requirements (NFRs); in the future, GPU-backed instances could be supported. These findings can also be used more generally, to optimise the balance between the computational resources needed and the resulting wall time to obtain results

Quality of Service-aware matchmaking for adaptive microservice-based applications

Applications that make use of Internet of Things (IoT) can capture an enormous amount of raw data from sensors and actuators, which is frequently transmitted to cloud data centers for processing and analysis. However, due to varying and unpredictable data generation rates and network latency, this can lead to a performance bottleneck for data processing. With the emergence of fog and edge computing hosted microservices, data processing could be moved towards the network edge. We propose a new method for continuous deployment and adaptation of multi-tier applications along edge, fog, and cloud tiers by considering resource properties and non-functional requirements (e.g., operational cost, response time and latency etc.). The proposed approach supports matchmaking of application and Cloud-To-Things infrastructure based on a subgraph pattern matching (P-Match) technique. Results show that the proposed approach improves resource utilization and overall application Quality of Service. The approach can also be integrated into software engineering workbenches for the creation and deployment of cloud-native applications, enabling partitioning of an application across the multiple infrastructure tiers outlined above

Towards a methodology for creating time-critical, cloud-based CUDA applications

CUDA has been used in many different application domains, not all of which are specifically image processing related. There is the opportunity to use multiple and/or distributed CUDA resources in cloud facilities such as Amazon Web Services (AWS), in order to obtain enhanced processing power and to satisfy time-critical requirements which cannot be satisfied using a single CUDA resource. In particular, this would provide enhanced ability for processing Big Data, especially in conjunction with distributed file systems (for example). In this paper, we present a survey of time-critical CUDA applications, identifying requirements and concepts that they tend to have in common. In particular, we investigate the terminology used for Quality of Service metrics, and present a taxonomy which summarises the underlying concepts and maps these terms to the diverse terminology used. We also survey typical requirements for developing, deploying and managing such applications. Given these requirements, we consider how the SWITCH platform can in principle support the entire life-cycle of time-critical CUDA application development and cloud deployment, and identify specific extensions which would be needed in order fully to support this particular class of time-critical cloud applications

Application-infrastructure co-programming: managing the entire complex application lifecycle

With an estimated 20 billion connected devices by 2020 generating enormous amounts of data, more data-centric ways of working are needed to cope with the dynamic load and reconfigurability of on-demand computing. There is a growing range of complex, specialised means by which this flexibility can be achieved, e.g. Software-defined networking (SDN). Specification of Quality of Service (QoS) constraints for time-critical characteristics, such as network availability and bandwidth, will be needed, in the same way that compute requirements can be specified in today's infrastructures. This is the motivation for SWITCH -- an EU-funded H2020 project addressing the entire lifecycle of time-critical, self-adaptive cloud applications by developing new middleware and tools for interactive specification of such applications. This paper presents a user-facing perspective on SWITCH by discussing the SWITCH Interactive Development Environment (SIDE) Workbench. SIDE provides a programmable and dynamic graphical modeling environment for cloud applications that ensures efficient use of compute and network resources while satisfying time-critical QoS requirements. SIDE enables a user to specify the software components, properties and requirements, QoS parameters, machine requirements and their composition into a fully operational, multi-tier cloud application. In order to enable SIDE to represent the software and infrastructure constraints and to communicate them to other SWITCH components, we have defined a co-programming model using TOSCA that is capable of representing the application's state during the entire lifecycle of the application. We show how the SIDE Web GUI, along with TOSCA and the other subsystems, can support three use cases and provide a walk-through of one of these use cases to illustrate the power of such an approach

Application-infrastructure co-programming: managing the entire complex application lifecycle

With an estimated 20 billion connected devices by 2020 generating enormous amounts of data, more data-centric ways of working are needed to cope with the dynamic load and reconfigurability of on-demand computing. There is a growing range of complex, specialised means by which this flexibility can be achieved, e.g. Software-defined networking (SDN). Specification of Quality of Service (QoS) constraints for time-critical characteristics, such as network availability and bandwidth, will be needed, in the same way that compute requirements can be specified in today's infrastructures. This is the motivation for SWITCH -- an EU-funded H2020 project addressing the entire lifecycle of time-critical, self-adaptive cloud applications by developing new middleware and tools for interactive specification of such applications. This paper presents a user-facing perspective on SWITCH by discussing the SWITCH Interactive Development Environment (SIDE) Workbench. SIDE provides a programmable and dynamic graphical modeling environment for cloud applications that ensures efficient use of compute and network resources while satisfying time-critical QoS requirements. SIDE enables a user to specify the software components, properties and requirements, QoS parameters, machine requirements and their composition into a fully operational, multi-tier cloud application. In order to enable SIDE to represent the software and infrastructure constraints and to communicate them to other SWITCH components, we have defined a co-programming model using TOSCA that is capable of representing the application's state during the entire lifecycle of the application. We show how the SIDE Web GUI, along with TOSCA and the other subsystems, can support three use cases and provide a walk-through of one of these use cases to illustrate the power of such an approach

Non-functional requirements optimisation for multi-tier cloud applications: An early warning system case study

Modern software engineering tools, technologies and approaches can radically speed up the development and engineering of multi-tier cloud applications and may be applied to the cloud/edge/fog computing paradigm. The engineering of such cloud applications must take into account Non-Functional Requirements (NFRs), such as end-user, software and cloud infrastructure requirements for low-power computing, performance, availability, elasticity, operational cost and similar. Such requirements should be identified and considered early in the software engineering process and decisions must be taken on their trade-offs, e.g. greater service availability while balancing operational costs. This paper introduces a new multi-criteria decision making approach via the use of the Pareto method to aid the software engineers with NFR trade-off adjustment possibilities. Adjustment possibilities may include geographic distribution of application tiers, horizontal and vertical scaling of virtual resources (such as Virtual Machines and containers) and similar. This work builds on top of technologies and tools developed under the SWITCH and ENTICE projects. The impact of this approach is the ability to fully tailor the adjustments of non-functional properties of the multi-tier cloud application according to the decisions of its engineer. An early warning system cloud application is used to present the approach

Quality of service models for microservices and their itegration into the SWITCH IDE

As demands on new software increase, new approaches are needed to help developers ensure Quality of Service (QoS) for their offered service. In this paper we present a QoS modeling approach that complements and extends the standard microservice and component-based software engineering tools by giving the software engineer information on what Non-Functional Requirements (NFRs) and quality constraints have critical influence on QoS. This concept extends the co-programing concept of programing the software and infrastructure with the ability to do this in a more informed manner. The result of this are Qualitative Metadata Markers (QMM) that provide guidelines for changing the infrastructure that can be used by humans or specialized decision-making algorithms

Support for full life cycle cloud-native application management: Dynamic TOSCA and SWITCH IDE

Motivated by the complexity and difficulty of engineering time-critical cloud applications, we investigated the methodology and software workbench to provide full application life cycle support to the software engineer. For this purpose we present a novel high-level concept that is concentrated on exchanging the complex, dynamic data using OASIS TOSCA standard, mainly used in static contexts. This methodology enables specifying an application logic, provisioning, deploying, monitoring, and dynamical adaptation to changing conditions. It can be applied effectively to the interchange of dynamically changing data in order to maintain Quality of Service. By extending TOSCA with dynamic mapping of Qualify of Service and runtime attributes related to the application and underlying architecture we show that TOSCA can be used to exchange data and reconfigure on-demand compute resources whilst the application is in the process of design, infrastructure provisioning, runtime or adaptation. To demonstrate that our novel research contributions can be realised, we implemented SWITCH IDE, which showcases this concept through its development workflow which offers application and underlying infrastructure description, manipulation and reconfiguration of QoS and runtime attributes. Our approach differs in that it supports the concept of dynamic TOSCA by directly enabling application runtime to be reconfigured on the fly